The Lancet Digital Health

Purpose To evaluate the external transportability of source-derived probability calibration for chest radiograph deep learning classifiers under distribution shift and determine its dependence on sample size and disease prevalence. Materials and Methods This retrospective secondary computational analysis of public, de-identified radiographic datasets involved no participant contact or new data collection. Cohort sizes were prespecified by availability without prospective power calculation: 106,686 NIH ChestX-ray14 images (29,323 patients; 1992-2015) for development and internal testing, 20,000 CheXpert images (20,000 patients; October 2002-July 2017), and 14,998 VinBigData images (2018-2020) for external testing. ResNet-18, DenseNet-121, and DeiT-Tiny classifiers for cardiomegaly and pleural effusion were trained across four patient fractions and three seeds (36 fits). Calibrators were fitted exclusively on NIH data. Twelve primary external Brier-score contrasts (temperature-scaled versus raw probabilities) were evaluated using paired cluster-bootstrapping with Holm adjustment. Results Temperature scaling reduced Brier scores in nine of 12 primary contrasts but worsened pleural effusion calibration on VinBigData across all architectures (Brier increases: ResNet-18, 0.001460 [95% CI: 0.001347, 0.001568]; DenseNet-121, 0.001193 [95% CI: 0.001109, 0.001274]; DeiT-Tiny, 0.000498 [95% CI: 0.000417, 0.000577]; all adjusted P=.006). Standardizing class weights to NIH prevalence reversed all CheXpert improvements. Although full versus 10% training improved external AUROC across all comparisons, DeiT-Tiny CheXpert pleural effusion Brier score worsened by 0.007039 despite a 0.032466 AUROC gain. At 80% selective image retention, calibrated ensembles retained only 12.3%-17.2% of VinBigData positive pleural effusion examinations. Conclusion Source-fitted calibration is not uniformly transportable across health systems, and discrimination gains do not guarantee calibrated probabilities. External evaluation must simultaneously assess probability reliability, prevalence shifts, and retained-case composition.

Background: Clinical registries and institutional quality control (QC) in pathology heavily depend on complex histopathological data. However, critical tumor characteristics are mostly locked in unstructured free-text reports. We hypothesize that locally deployed, resource-efficient language models operating within a secure institutional network can reliably extract relevant variables from German reports of hepatocellular carcinoma (HCC), thereby enabling automated data curation without transferring sensitive patient data. Methods: We developed a local, privacy-preserving open-weight Large Language Model (LLM) pipeline consisting of two consecutive stages: Stage 1 utilizes a lightweight gatekeeper model to isolate the target cohort of confirmed Hepatocellular Carcinoma cases from a screening pool of N = 2,864 reports. Stage 2 deploys a high-capacity model to extract 15 predefined histopathological variables, including tumor-node-metastasis staging, histological grading, vascular invasion, and selected immunohistochemical features into structured JSON format. For both Stages, pipeline performance was evaluated using Accuracy, Macro F1-scores, and Cohen's kappa compared to expert-curated reference annotations. Results: The top-performing Stage 1 gatekeeper isolated the target patient cohort with zero classification errors, effectively reaching complete agreement with the reference standard (ground truth) and correcting inaccurate administrative metadata. For Stage 2 feature extraction, current-generation models demonstrated robust structural reliability, reducing the burden of manual post-processing. The 31B model configuration (Gemma 4) virtually eliminated objective factual errors, achieving near-perfect agreement with the reference standard (Macro F1 > 0.99). Conclusion: Local two-stage pipelines offer a highly accurate, resource-efficient, and secure solution for clinical text processing. This framework serves as a privacy-preserving engine for institutional quality control and the scalable curation of pathology registries.

Background: Many radiology decisions, such as whether a report describes a finding or how urgently a study should be read, are choices among a fixed set of answers. Decision models return a probability for each permitted answer in one step, without generating text, and an open-weight model can run within a hospital so that patient data stay local. No decision model has yet been specialized for radiology and compared with large language models (LLMs). Methods: We fine-tuned the general-purpose decision models Kev-27B (Qwen3.8-27B backbone) and Kev-9B on 36,109 records from public radiology datasets to obtain RadKev-27B and RadKev-9B, and compared them with the Kev models, the hosted decision models Jev and OpenAI Decisions, and the LLM Qwen3.8-27B, on 14,142 held-out questions in fifteen tasks (report reading, case diagnosis and classification, radiological knowledge). The prespecified primary outcome was task-averaged accuracy relative to Kev-27B. Results: RadKev-27B exceeded Kev-27B by 4.9 percentage points (95% CI 3.8 to 6.2), with the largest gain in report error detection (22.6 points); RadKev-9B exceeded Kev-9B by 6.9 points (5.5 to 8.4) and did not differ detectably from Kev-27B, a model three times its size. RadKev-27B was more accurate than the hosted decision models Jev (3.0 points, 1.7 to 4.4) and OpenAI Decisions (4.8, 3.2 to 6.4), than Qwen3.8-27B without reasoning (4.7, 3.4 to 6.0) and with reasoning (2.8 points, 1.0 to 4.6) on 1,675 questions, at 132-fold lower latency (128 ms vs 16.9 s per question). At an error rate of 5%, RadKev-27B could answer 90.5% of questions and Kev-27B 76.4%. Conclusions: RadKev-27B was more accurate than two hosted decision models and an LLM of the same size, with or without reasoning, at a fraction of the latency of reasoning, and can be deployed within an institution for categorical radiology decisions after validation on local data.

Background: General-purpose large language models (LLMs) are increasingly evaluated in diagnostic pathology, but prior studies have largely emphasized diagnostic accuracy rather than how models fail. We evaluated four LLMs for diagnostic performance, pathology-relevant errors and hallucinations, their burden, and potential clinical impact across multi-organ pathology cases. Design: In this retrospective multicenter study, 153 pathology cases from two institutions spanning 20 organs were evaluated using ChatGPT-5.3 (LLM1), Gemini 3 (LLM2), Grok 4.20 (LLM3), and Claude Opus 4.6 (LLM4). Each LLM received multi-magnification histologic images with clinical context and generated a microscopic description and diagnosis. No data splitting, training, or fine-tuning was performed. Twenty-one pathologists assessed 612 outputs for diagnostic correctness, error and hallucination type and burden, clinical impact (0-4), and overall performance (1-5). Results: Strict diagnostic accuracy was 48.9% overall (60.1% including partially correct diagnoses) and ranged from 36.6% to 58.2% across LLMs. Errors occurred in 82.0% of outputs and hallucinations in 76.6%; 90.2% contained at least one error or hallucination. Misinterpretation was the most frequent error (72.1%), while fabricated histologic features were the dominant hallucination type (74.5%). LLM2 had significantly lower misinterpretation rates than the other three LLMs, while LLM3 had significantly higher fabricated-feature hallucination rates than LLM1 and LLM2. Errors were more frequent in incorrect than correct diagnoses (97.5% vs 66.2%), as were hallucinations (95.1% vs 59.9%; both p < 0.001). Every strictly incorrect diagnosis contained an error and/or hallucination, while 79.9% of strictly correct diagnoses also contained at least one. In multivariable analysis, LLM3 was independently associated with higher odds of an incorrect diagnosis. Diagnostic correctness was the strongest determinant of high clinical impact; each one-point increase in hallucination burden increased the odds by 63% (adjusted OR, 1.63). Conclusion: Diagnostic accuracy alone substantially underestimates the safety limitations of general-purpose LLMs in histopathology. Errors and hallucinations were common even when the final diagnosis was correct, and their burden was independently associated with clinical impact. Evaluation frameworks should therefore assess both diagnostic correctness and the reliability and potential consequences of accompanying generated content.

On the public Regensburg Pediatric Appendicitis Dataset, several recent studies report near-ceiling diagnostic performance using radiologist-interpreted ultrasound findings as tabular model inputs, without processing the underlying images. This raises a question worth answering carefully: how much of that performance reflects genuine diagnostic information, and how much reflects the way this dataset's outcome labels were constructed? We defined four modeling arms, an image-only arm (a backbone-matched BiomedCLIP pipeline as the primary comparator, with an originally-planned ResNet18 model retained as a secondary baseline), a clinical/laboratory-only model (XGBoost), a radiologist-findings model (Random Forest, treated as a reference built from expert-recorded features rather than a live expert comparator), and a multimodal model fusing frozen BiomedCLIP image embeddings with clinical/laboratory data, and compared them using paired DeLong tests, bootstrap confidence intervals, TOST equivalence testing, and pooled repeated cross-validation. The multimodal model (pooled AUC 0.894) showed significantly higher discrimination than the backbone-matched image-only (0.789, p less than 0.0001) and clinical-only (0.855, p less than 0.0001) models. Against the radiologist-findings model (0.934), it initially fell short (p = 0.034). We then found that the dataset's own rule for labeling non-surgically-managed patients uses appendix diameter, one of the radiologist-findings predictors, and, separately, the Alvarado and Pediatric Appendicitis Scores, both retained as clinical/multimodal predictors. Removing appendix diameter dropped the radiologist-findings AUC to 0.790; removing the two scores from the multimodal model changed its AUC only marginally, from 0.894 to 0.890, a difference not statistically distinguishable from no change (p = 0.25). Comparing these two corrected models directly, the multimodal model remained significantly higher (0.890 vs. 0.790, p = 6.2 times 10 to the power of minus 6). A supplementary check found that 137 of 578 patients (24 percent) have ultrasound images containing burned-in caliper measurements; on a matched comparison holding the patient set fixed, excluding these images significantly reduced both the image-only and multimodal models' AUC, indicating a second, image-level information pathway that our design cannot fully attribute to on-screen numeric values versus other differences between marked and unmarked views. An exploratory check that combined both corrections, restricting the doubly-corrected multimodal model to annotation-free images on the smaller, matched patient subset, found a smaller, non-significant difference against the diameter-removed radiologist-findings model (p = 0.176), so our main doubly-corrected comparison should not be read as controlling for this second pathway. The multimodal model showed higher discrimination than the unimodal automated baselines on this benchmark, significantly so in pooled repeated cross-validation; its apparent shortfall against radiologist-derived features is highly sensitive to the dataset's outcome-definition procedure, although the exact fraction of that shortfall attributable to label-construction dependence, as opposed to genuine diagnostic information, cannot be determined from these data alone.

Background. Chest radiographs (CXRs) are the most common imaging exam performed, but CXR reports can be nonspecific due to incomplete history, relying on vague terms like opacities to convey diagnostic uncertainty. We prospectively evaluated whether introducing a clinical dashboard with relevant patient information affects CXR report specificity and efficiency, report concordance with discharge diagnoses, and follow-up chest CT utilization. Methods. In this stepped-wedge cluster randomized trial (NCT06785246), a clinical dashboard for radiologists was rolled out with user education sessions across 15 service areas of an integrated health system from June 2024 to June 2025. Emergency department (ED) and inpatient CXRs were included. A large language model (GPT-4o) was validated using a Delphi panel of three experts to classify report specificity and CXR report diagnosis. A subsequent model release (GPT-5.4) graded each CXR report impression as negative, specific abnormal, or nonspecific abnormal. It also judged whether an abnormal impression matched an ED discharge diagnosis. The primary outcome was the proportion of specific reports (negative or specific abnormal). Secondary outcomes were report-discharge diagnosis concordance, follow-up chest CT, antibiotic and diuretic orders, and CXR reporting time. Mixed-effects models adjusted for time, patient age, and radiograph view, with random effects for radiologists, facilities, and service area. Results. The trial included 404,860 ED and inpatient encounter STAT priority CXRs read by 387 radiologists (256,303 control; 148,557 intervention). The adjusted proportion of specific reports was 82.2% in control periods and 82.1% in intervention periods (difference, -0.10 percentage points; 95% CI, -0.53 to 0.34; P=.66). Among abnormal reports, concordance with a discharge diagnosis rose from 40.9% to 42.2% (difference, 1.24 points; 95% CI, 0.20 to 2.28; P=.02). Follow-up chest CT fell from 11.4% to 10.9% (difference, -0.55 points; 95% CI, -0.92 to -0.17; P=.004). Antibiotic orders (24.0% vs 23.9%) and diuretic orders (14.7% vs 14.8%) within 7 days did not change, nor did reporting time (difference, -0.8 seconds; 95% CI, -2.7 to 1.0). Conclusions. Giving radiologists EHR context through a dashboard did not change the specificity of STAT priority CXR reports or prescribing patterns. It was associated with small improvements in diagnostic concordance and follow-up CT use, without slowing reporting. Validated large language models make it feasible to measure report quality across a health system.

Background Artificial intelligence computer-aided diagnosis (AI CAD) algorithms for screening mammography have shown promise, but independent head-to-head comparisons of commercial algorithms on large, diverse cohorts remain limited. Methods 786,124 mammography screening exam performed from January 1 2022 to December 31 2023 were identified from electronic medical records. Three FDA-cleared commercial algorithms (A, B, D) and the open-source academic model Mirai (C), were applied to the four standard screening views. Breast cancer within 12 months was ascertained through cancer registry linkage. Algorithms were compared by AUROC; sensitivity, specificity, and PPV at matched AI-positive rates of 5%, 10%, 15%, 20%, 25%; and the proportion of false negatives classified as AI-positive. Results Of 707,922 examinations with complete data, 4,436 were associated with cancer. Radiologists recalled 6.8% of examinations, with a sensitivity of 67.2%. Algorithm D had the highest AUROC (0.849), followed by algorithm B (0.818), Mirai (0.817), and algorithm A (0.815). Algorithm D also had the highest sensitivity at every AI-positive rate, from 55.3% (95% CI: 54.1, 56.6) at 5% to 77.9% (95% CI: 76.9, 79.0) at 25%, with CIs that did not overlap those of the other algorithms. No algorithm matched radiologist sensitivity at AI-positive rates of 10% or less. Algorithm B flagged a proportion of radiologist false negatives similar to that of algorithm D from 5% through 20% (20.8% vs 20.5% at 5%), despite its lower standalone sensitivity. Conclusions AI CAD algorithms differed meaningfully in performance on the same large, diverse screening examination cohort, and an open-source academic model performed comparably to two of the three commercial products. Independent local validation at clinically relevant operating points should precede adoption of mammography AI, and algorithm choice and threshold should be matched to the intended workflow.

Objective: To determine how cohort versus nested case-control (NCC) design affects EHR-based prediction of a rare outcome. Materials and Methods: We compared two cohort and five NCC designs for predicting early-onset gastrointestinal cancer using L1-regularized logistic regression (LASSO), XGBoost, and multilayer perceptron (MLP). Models underwent nested five-fold cross-validation with model-specific hyperparameter tuning and temporal evaluation in a held-out prospective cohort representing intended deployment. Model discrimination was assessed using area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC). Model calibration was assessed using Brier scores and calibration plots before and after recalibration with isotonic regression. Feature-ranking similarity was quantified using rank-biased overlap. Results: The cohort development datasets included 256,854-415,529 patients, with 2-yr cumulative incidence of 0.12%-0.17%, compared with approximately 9% in the 1:10 matched NCC datasets. NCC designs generally produced higher internal discrimination, but temporal performance varied by algorithm and index-date definition. Cohort designs achieved the highest temporal AUPRC for LASSO (0.068), whereas diagnosis-anchored NCC achieved the highest temporal AUPRC for XGBoost (0.086) and MLP (0.070). Temporal AUPRC generally decreased as the required lead time before diagnosis increased. Raw predicted probabilities substantially overestimated observed risks in the temporal cohort. Isotonic recalibration improved probability scaling while largely preserving discrimination, although residual miscalibration remained. Cohort-based feature rankings were more consistent across algorithms, whereas NCC-based rankings depended more strongly on algorithm and index-date anchoring. Conclusion: Study design should be selected and evaluated according to the intended clinical prediction task and deployment setting.

BackgroundRoutinely collected health data can reveal recorded care around diagnosis, but disease-specific curation does not scale across thousands of conditions. We scaled a previously developed automated framework that reduces heterogeneous clinical events into standardised, diagnosis-centred summaries of real-world care patterns. MethodsIn this retrospective observational study, we used EST-Health-30, a pseudonymised 30% random sample of Estonian residents with health-care records from 2012 to 2024. We constructed cohorts for ICD-10 three-character diagnostic categories using the first eligible observed diagnosis after a 3-year diagnosis-free observable lookback. For each category, we summarised recorded clinical events from a 90-day pre-index window, a 30-day post-index window, and a 365-day post-index window. An enrichment-based workflow compared earlier self-comparator periods and matched population controls, then filtered and aggregated concepts across clinical domains. Diagnosis-concept relationships were assessed by rubric-guided language-model classification, and atlas plausibility and utility by ten experts. FindingsOf 1645 observed diagnostic categories, 1080 met inclusion criteria, yielding 3240 diagnosis-window summaries across 509,856 individuals. Each summary characterised diagnosis-associated events by prevalence, enrichment, timing, co-occurrence, and variation between patient groups after reducing candidate concepts by 97-98%. Among 296,303 retained diagnosis-concept pairs, the language model classified 24% as directly related, 55% as indirectly related, and 21% as noisy. Experts rated the atlas highly for generating research questions (median 6{middle dot}5 of 7 [IQR 6-7]), providing information difficult to obtain conventionally (6 [5-7]), and supporting observational study design (5{middle dot}5 [5-6]). InterpretationThe atlas makes multidimensional diagnosis-centred patterns in longitudinal health records inspectable at population scale in order to support clinical orientation, hypothesis generation, and observational study planning. Outputs represent recorded care rather than individual patient trajectories or causal effects. FundingEstonian Research Council; European Union; Estonian Ministry of Education and Research; Innovative Medicines Initiative 2 Joint Undertaking; EFPIA. Research in contextO_ST_ABSEvidence before this studyC_ST_ABSWe searched PubMed and medRxiv from database inception to Aug, 2026, without language restrictions, using terms for electronic health records, real-world data, disease trajectories, treatment and care pathways, treatment patterns, clinical-event analysis, OMOP, and population-scale atlases. We also examined relevant reviews and reference lists. Previous population-scale studies have mapped disease trajectories, treatment pathways, or care intensity, generally focusing on diagnoses alone, prespecified events, selected disease cohorts, or a single summary measure of care. OMOP-based treatment-pattern studies have likewise concentrated on specific disease groups. We found no previous population-scale resource that automatically reduced multidomain clinical events surrounding an index diagnosis across more than 1000 diagnostic categories while retaining prevalence, enrichment, timing, co-occurrence, and patient-level heterogeneity. Added value of this studyWe applied a previously clinically benchmarked concept-reduction workflow to longitudinal OMOP-mapped health data across 1080 ICD-10 diagnostic categories and three observation windows, producing 3240 diagnosis-window summaries from 509,856 individuals. The workflow reduced candidate clinical concepts by 97-98% and generated compact summaries spanning diagnoses, procedures, medications, measurements, observations, visits, and deaths. The resulting interactive atlas combines concept prevalence and enrichment with timing, co-occurrence, and exploratory patient clustering. An internal formative expert evaluation supported its use for generating research questions and planning observational studies. Implications of all the available evidenceAutomated reduction of high-dimensional health-record data can provide a scalable starting point for clinical orientation, hypothesis generation, and observational study design without requiring a separately curated event set for every disease. Applying the same workflow to other OMOP databases could enable direct comparison of diagnosis-centred treatment and care patterns between regions and health systems, while helping distinguish differences in clinical practice from differences in data capture. The atlas is exploratory and should not be interpreted as evidence of causal effects, guideline adherence, or individual patient trajectories.

Fairness audits of chest radiograph artificial intelligence usually score subgroup performance against labels extracted from radiology reports by natural language processing (NLP). We tested whether these labels can make a model look fairer than it is, a failure we call phantom fairness. On 4,376 NIH ChestX-ray14 images with both NLP and radiologist-adjudicated labels, we audited classifiers from three backbones by sex, age, projection and older women. NLP labels missed 54% to 59% of radiologist-confirmed airspace opacity, pneumothorax and nodule or mass, and every fracture. These misses were not random: models scored them lower than NLP-caught positives for every finding and backbone (median rank-biserial correlation 0.20 for the primary backbone and 0.16 for the others; all adjusted p [&le;] 0.038), so the labelling errors hid the models' own mistakes. Subgroup gaps were larger on adjudicated labels in 16 of 24 comparisons, none individually significant. For pneumothorax in women older than 60 years, the NLP audit showed a sensitivity gap of 0.064 against a true gap of 0.184; removing the hardest positives reproduced this concealment (0.124 against 0.120 observed), whereas random label errors produced less (0.077). Fairness audits based on NLP-extracted labels should be treated as provisional until checked against radiologist-adjudicated labels.

Frailty assessment is increasingly recommended for risk stratification in chronic disease management, but systematic clinical screening for frailty remains difficult to implement because it is resource-intensive. We investigated whether routinely collected clinical measures could be repurposed to prioritize adults with chronic diseases for clinical frailty assessment. We conducted a prediction study using UK Biobank data. A stacked ensemble model was developed using 12 routinely and low cost measured clinical variables to predict frailty. Gain metrics were used to quantify the proportion of frailty cases identified when screening was restricted to individuals with the highest predicted risk of frailty. We compared health profiles (mortality, polypharmacy, self-rated health, falls) of frail and non-frail individuals across different levels of predicted risk. The robustness of this screening strategy was evaluated across 3,284 modelling settings. Among 281,729 participants (mean age 57.9; 54.7% women) with at least one chronic condition, 5.8% were frail. In the validation sample (N validation=93,910, N training=187,819), screening only the 20% of participants with the highest predicted risk of frailty identified 55.6% of all frailty cases, corresponding to a 2.78-fold improvement compared to random screening. Overall, the model recovered 53.4% of the maximum achievable improvement over random screening. Frailty remained always associated with poor outcomes. Screening performance remained consistent across alternative modelling settings. Routinely collected clinical measures can improve the efficiency of frailty screening among adults living with chronic diseases by prioritizing individuals based on their level of risk. Since these data are already gathered during routine care, this risk-prioritization strategy could support scalable screening without increasing testing burden or clinical workload.

Rationale: Obstructive sleep apnea (OSA) is highly prevalent yet largely under-diagnosed. Current screening strategies rely on effortful questionnaires with modest accuracy, and prior machine learning models often require resource-intensive inputs and lack external validation. Objectives: To develop and validate machine learning models to predict OSA (apnea-hypopnea index [AHI4%][&ge;]5) and moderate-severe OSA (AHI4%[&ge;]15) using only routinely collected electronic health record (EHR) data, test cross-system transferability and compare performance with clinical screeners and a Bayesian risk-updating framework. Methods: Three tree-ensemble families (XGBoost, random forests, and histogram-based gradient boosting) were trained on an adult cohort with sleep-study results at Kaiser Permanente (KPSC), using full (106-feature) and minimal (age, sex, body-mass index, race/ethnicity) predictor sets and tuned by nested cross-validation. XGBoost models were selected, internally and temporally validated, and externally validated at University of Kansas Medical Center (KUMC). Measurements and Main Results: On our internal hold-out set (n=53,048), the full XGBoost model reached a ROC-AUC of 0.791 for any OSA and 0.768 for moderate-severe OSA. The minimal model retained most performance (0.764 and 0.736). Temporal performance was stable. Externally, frozen models showed lower AUCs (0.718 and 0.690 for full models; 0.725 and 0.706 for minimal models) but similar to KUMC-native models and improved further with transfer learning. The models significantly outperformed STOP-BANG (AUC difference 0.06-0.09 for predicting any OSA; p<0.001). Bayesian updating with select patient-reported items further improved discrimination. Conclusions: Machine learning models based on EHR data can predict OSA probability that outperform existing screeners, transfers across systems, and offers potential for EHR implementation.

Background: Conventional performance metrics do not indicate whether artificial intelligence (AI)-platforms and radiologists are right or wrong on the same examinations, nor whether their non-agreement is more often beneficial or harmful. Objective: To develop, illustrate, and apply directional metrics quantifying non-agreement between radiologists and AI-platforms against a common reference standard, and to adapt a fragility-based framework for quantifying the statistical robustness of these metrics. Methods: We defined Positive Non-Agreement (PNA, AI-platform correct, radiologist incorrect), Negative Non-Agreement (NNA, radiologist correct, AI-platform incorrect), and Net Non-Agreement (NENA, their difference). Metrics were illustrated in three hypothetical scenarios of 1,000 examinations each and applied to three previously published AI-radiologist studies on pulmonary embolism and mammography. Bootstrap confidence intervals were calculated, and a reclassification fragility index (FI) quantified the minimum case reclassification needed to remove statistical support for NENA's direction. Results: In illustration, NENA was positive in two scenarios and negative in one. In the three published studies, NENA ranged from -4.18% to +10.74% and decreased consistently with increasing radiologist experience (junior residents, +6.27%; consultants, +1.45%), a gradient driven primarily by declining PNA with modest increase in NNA. The reclassification FI was defined for all three illustrative scenarios (range, 0.10%-1.70%) and for six of the seven comparisons from these studies (range, 0.34%-4.31%), but not for the weakest comparison, whose confidence interval for NENA included zero. Conclusions: PNA, NNA, and NENA provide a directional, case-level complement to conventional performance metrics for AI-radiologist non-agreement. The accompanying reclassification FI offers a statistically grounded, generalizable method for quantifying their robustness.

Background: Patients increasingly consult free artificial intelligence (AI) chatbots about their symptoms, but these models may favour agreement with the patient over accuracy. In obstructive sleep apnoea (OSA), where most cases in the general population remain undiagnosed and diagnosis depends on referral, this could validate symptom denial. We tested whether consumer chatbots maintain guideline-concordant referral when a simulated patient minimises symptoms. Methods: Seven guideline-concordant and standardised OSA vignettes (apnoea-hypopnoea-index, AHI 8-52 events/h; body mass index 28-42 kg/m2) were presented to five free chatbots (GPT-5, Gemini 2.5 Flash, Claude Sonnet 4.6, DeepSeek V4 Flash and Grok-4.3) under neutral and symptom-minimising conditions (10 simulated conversations per model-vignette-condition cell; N=700). Referral maintenance was scored by a structured rubric and every capitulation was human-confirmed. Results: All chatbots maintained referral recommendations in neutral conversations (350/350; 100%). When simulated patients minimised symptoms and resisted referral, maintenance fell to 64.3% (225/350), a reduction of 35.7 percentage points (95% CI 5.5-66.0; p=0.028). Rates ranged from 55.7% to 85.7% across chatbots, but evidence of between-chatbot differences was inconsistent across analyses. Rates differed across clinical vignettes (p=0.001). A variance analysis estimated greater variation between vignettes than between chatbots. In the three driving-risk vignettes, referral recommendations were abandoned in 46-68% of symptom-minimising conversations, pooling all five chatbots within each vignette. Conclusions: Consumer AI chatbots abandoned correct OSA referral recommendations in about one-third of symptom-minimising conversations, with higher observed abandonment in simulated cases involving driving risk. This may delay appropriate assessment and raises patient- and public-safety concerns.

Background: Identification of in-hospital cardiac arrest (IHCA) through manual chart abstraction is the gold standard, but its time-consuming and resource-intensive, which naturally limits its practicality. Diagnostic codes provide an accessible automated alternative, but prior work has shown this method of extraction has poor sensitivity and poor positive predictive value. More accurate and scalable automated methods are needed to support IHCA surveillance and research. Methods: We developed large language model (LLM)-based systems that incorporated physician-defined clinical knowledge through guideline-enriched prompting, supervised fine-tuning, and a self-correction procedure. The systems identified IHCA events and their locations from electronic health record notes. Performance was evaluated at the encounter level against physician-adjudicated chart review and compared with ICD-code-based identification. Results: Compared with an ICD-code-based algorithm (precision 0.68, recall 0.91, F1 score 0.78), the supervised fine-tuned LLM offered substantially better performance for IHCA identification on a curated evaluation corpus, reaching a precision of 0.88, a recall of 1.00, and an F1 score of 0.94. When tested on a larger, unselected validation cohort of 45,525 encounters, a separate LLM configuration - relying on guideline-enriched prompting and a self-correction procedure rather than fine-tuning - achieved a precision of 0.79 for encounter-level IHCA identification, meaning most encounters flagged by the model were genuine IHCA cases. The fine-tuned LLM underperformed on this cohort due to its inability to self-correct its decisions, an issue that can be mitigated by having a separate model perform the correction step. Conclusion: Explicit integration of clinical knowledge through prompting and supervised fine-tuning enabled accurate identification of IHCA events and locations from routine clinical documentation. In a two-pass workflow, these systems could prioritize encounters for clinician adjudication, reduce manual screening requirements, and support more scalable IHCA surveillance and registry development.

INTRODUCTION: We report the development and validation of a novel computational tool to cross-convert plasma pTau217 results from different assays to facilitate recruitment for therapeutic trials and observational studies. METHODS: Participants underwent A{beta} PET imaging and plasma pTau217 measurement on up to three assay platforms across three cohorts. We then developed conversion equations using Deming regression -- collectively named Biomarker Convert -- and evaluated robustness using independent research cohorts. Finally, we modeled the potential impact of pTau217 screening on trial enrollment. RESULTS: While absolute plasma pTau217 concentrations differed, the assays had similar predictive characteristics for A{beta} positivity (LC-MS/MS [C2N], AUC=0.952; Janssen Simoa, AUC=0.946; Lumipulse, AUC=0.929). Biomarker Convert enabled transformation of plasma pTau217 data across assays independent of A{beta} PET and clinical status. Plasma pTau217 reduced A{beta} PET scans in the model screening paradigm by >60%. DISCUSSION: Biomarker Convert equations enable harmonization of plasma pTau217 values from immunoassay and mass spectrometry platforms, supporting direct data comparison.

Background: Machine Learning (ML) has shown promise in histopathological diagnosis, but rare tumours, like duodenal epithelial neoplasms, are challenging because large, labelled datasets are difficult to assemble. We propose a data-efficient ML pipeline for detecting duodenal epithelial neoplasia in whole-slide images. Methods: We assembled a five-hospital dataset of haematoxylin and eosin-stained duodenal biopsy whole slide images (WSIs) at 10x magnification, comprising 8,027 normal and 401 neoplastic biopsies, including adenomas, carcinomas and neuroendocrine tumours. WSIs were divided into patches and encoded using the Hibou-B foundation model. We compared global standardisation with unsupervised site-specific standardisation of patch embeddings. Our pipeline combined principal component analysis, a Gaussian mixture model and a Random Forest classifier. We assessed generalisability using leave-one-hospital-out cross-validation across four hospitals and on an independent test set from a fifth hospital. Results: In leave-one-hospital-out cross-validation, the pipeline achieved a mean accuracy of 90%, sensitivity of 96%, specificity of 83% and an AUC of 0.96. On the external test hospital, it achieved 94% accuracy, 93% sensitivity, 95% specificity, and AUC of 0.97. Site-specific standardisation improved mean cross-validation accuracy by 20 percentage points compared with global standardisation, while the Gaussian mixture pipeline improved accuracy by 12 percentage points compared with mean pooling of patch embeddings. Conclusions: We present a data-efficient ML approach for detecting duodenal epithelial neoplasia from WSIs. Strong performance was maintained across hospitals and on an independent external test cohort despite the limited number of neoplastic training cases, supporting the feasibility of foundation-model-based approaches for rare histopathological diagnoses.

Background Cardiovascular prevention is organized around target metrics. Blood pressure, LDL cholesterol, and A1c define treatment targets, quality metrics, and, implicitly, who is considered safe. Yet a value below a cutoff is a correlative measurement, not an outcome. We tested whether social context recorded before the measurement separated subsequent cardiovascular events among people whose numbers already met the cutoff. Methods In this retrospective cohort study, we analyzed All of Us participants without recorded cardiovascular disease and with low comorbidity in three separately indexed cohorts: systolic blood pressure (SBP), LDL cholesterol, and A1c. A preindex score combined household income, education, food security, physical activity, and ZIP-linked deprivation, standardized against a fixed pooled reference of 325,652 eligible people. The outcome was first myocardial infarction, ischemic stroke, or heart failure. A sensitivity analysis indexed participants at only events recorded in inpatient or emergency settings. Results The SBP, LDL, and A1c cohorts included 19,525, 16,578, and 18,539 participants with 144, 129, and 191 events. Each one-SD more favorable score was associated with lower hazard: HR 0.611 (95% CI 0.498-0.749), 0.631 (0.512-0.778), and 0.544 (0.457-0.648), respectively. Below each study cutoff (SBP <140 mmHg, LDL <100 mg/dL, A1c <8%), less favorable context had event rates of 3.39 versus 1.68, 7.16 versus 2.05, and 8.67 versus 2.34 per 1,000 person-years. In a one-year analysis restricted to 2019-2023 index dates, the corresponding adjusted HRs were 0.530, 0.336, and 0.365. In the sensitivity cohorts (54,338, 41,451, and 33,607 participants; 493, 288, and 316 events), HRs were 0.732 (0.657-0.815), 0.660 (0.573-0.761), and 0.631 (0.552-0.722). Conclusions A biomarker below the study cutoff did not identify a uniform, low-risk population. Social context recorded before the measurement separated cardiovascular events after adjustment for the measured value and clinical factors. Risk assessment that stops at the number misses this variation.

Background: Patients with intracerebral hemorrhage (ICH) in intensive care units (ICUs) frequently receive antibiotics early in critical illness. Such exposure reflects suspected infection, physiological deterioration, monitoring intensity, and clinical management and may vary across institutions and electronic health record databases. We evaluated target-domain adaptation for modeling sustained antibiotic exposure in critically ill patients with ICH. Methods: We analyzed derived ICH cohorts from MIMIC-IV and the eICU Collaborative Research Database. MIMIC-IV included 2,024 ICU stays and served as the source cohort. The eICU cohort comprised 1,277 stays and was divided, at the ICU-stay level, into a development subset (766 stays) and a held-out evaluation subset (511 stays). The eICU outcome was sustained antibiotic exposure on at least two ICU days. Candidate models incorporated demographic, comorbidity, physiological, laboratory, neurologic, respiratory, temporal, and measurement-process features. Timestamped observations for static and temporal tabular families were censored at the recorded first-antibiotic time when available, whereas the compact sequence component and explicit measurement-density features used the full first 24 hours. For base-model families using eICU development data, meta-learner inputs were generated by five-fold stratified cross-fitting. A 42-input L2-penalized logistic meta-learner combined 16 model probabilities with model-summary, clinical, availability, and measurement-density features. Results: The held-out eICU subset contained 48 positive outcomes (9.39%). The adapted ensemble achieved an area under the receiver operating characteristic curve (AUROC) of 0.841 (95% CI 0.775-0.899) and an average precision (AP) of 0.540 (95% CI 0.399-0.676), with a Brier score of 0.063 and expected calibration error of 0.029. The MIMIC-IV source-only stack achieved an AUROC of 0.782; the paired AUROC difference was 0.059 (95% CI 0.003-0.112). In a separate patient-grouped five-fold out-of-fold sensitivity analysis of 503 linked eICU stays, AUROC was 0.807 (95% CI 0.691-0.910). Conclusions: Target-domain adaptation improved held-out discrimination relative to a source-only stack for sustained antibiotic exposure. Because some model components and measurement-process features used information from the broader first 24 hours, the model should be interpreted as a first-24-hour treatment-association model rather than a strictly prospective pre-treatment infection predictor.

Current state of the art computational pathology foundation models attain satisfying accuracy most general pathology tasks including detection of pancreatic ductal adenocarcinoma (PDAC) but fail to provide sufficient explainability, operating as "black boxes". Some interpretability effort has been made, with models providing attention maps that project back on the slide the models high attended areas that pushed for the given prediction result, without further naming or reasoning explanation which can limit pathologists trust toward the prediction. We developed C3PRO, a cell-centric framework attempting to provide deeper understanding and explainability, by using embeddings extracted using a pathology foundation model, from cell centered tiles containing both the cell and its close neighborhood, which were then clustered into recurrent morphological phenotypes. A spatial graph was additionally constructed on top of the back-projected vocabulary to account for recurrent motifs. The evaluation of the model performance involved a cohort of 809 patients (825 diagnostic biopsies) using nested cross validation. C3PRO was benchmarked against four different multiple instance learning frameworks and achieved on-par performance for diagnostic (0.958 AUC), survival (0.617) and resectability (0.696) prediction, while providing additional insights compared to the strongest baseline models. Each clustered phenotype was probed and extracted as a representative tile mosaic and annotated by two expert pathologists, constituting a phenotype vocabulary available for statistical analysis. These phenotypes were scored by pathologists with physical brightfield microscopy and replicated the independent statistical significance impact on overall survival prediction from diagnostic biopsies.